Improvement of an Online Music Alignment Based on Onset Information

Music alignment is the association of events in a musical score with points in the time frames of an audio signal. So it is also called audio-score alignment. Until now, most polyphonic audio-score alignment methods are offline algorithms, and alignment accuracy drops significantly when using online algorithms. But most applications based on alignment need real-time results, such as Automatically Accompaniment System or Music Tutor. So we aim at improving the alignment result of an online alignment algorithm. In general online methods, a polyphonic music audio is segmented into time frames and they are fed to the score follower in sequence. Then the algorithm outputs a score position for each frame right after it is processed. In this paper, we find that not all audio frames give us the same level of reliability within the evolution of one note, especially in staccato performing or decay of piano sound when real sound will last shorter compared with our expectation from the score. Then the online follower in previous algorithm will get lost. So in this paper, we propose to utilize note onsets information to find the “faithful” frames as observation to do the alignment. A weighting function is introduced to assign different weights to the frames as the measure of their reliability. The function value is dropping within evolution of one note and score position tends to move forward evenly excluding observations from audio, while when a new onset is detected and weight returns high, it will be dragged to right position quickly due to high trust of observation. Experiments show that proposed improvements can get better alignment results than previous online methods, especially in piano music.